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ABSTRACT 


To improve the learning process, the evolution of learner’s characteristics (cognitive, affective, prior knowledge, 
workflow, organization, ...) must be taken into account during the personalization or adaptation. This requires generating 
several scenarios (a description of activities, their order and links in the learning sequence as well as the expected 
outcome for the learner) adapted to the identified profiles. We propose a model which aims at improving learners’ 
learning processes by giving them control over two key aspects: (1) the steps of the learning scenario to be followed: after 
each learning goal is completed, the learner chooses the next one among the possible ones (in terms of their current 
knowledge) while respecting pedagogical constraints (time and quality of the solutions produced according to satisfaction 
thresholds); (2) the assessment mode: the learner chooses a mode corresponding to their own goals in terms of mastery, 
while respecting the minimum thresholds set by the teacher. We assess our approach with learners in terms of 
(a) adequacy of the model with learners’ expectations, (b) usability of the system and (c) learning experience satisfaction, 
through self-report questionnaires and an analysis of the data collected over 11 learners who used an implementation of 
our system on the LMS (Learning Management System) in the context of a real course on Economy. The results reveal an 
a priori acceptance of our model, a diversity of the scenarios constructed, and the use of 2 (out of 3) assessment modes to 
progress. We use these results to analyze current limits of the system and propose redesign ideas to minimize them. 
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1. INTRODUCTION 


The teacher creates a knowledge-based course with well-defined learning goals. The course is then organized 
in a scenario which guides teaching and learning. This standard scenario, as envisioned by the teacher, can be 
inappropriate or at least suboptimal for some learners, because the learning also depends on their personal 
characteristics (e.g. pace of work, cognitive styles, emotional factors, prior knowledge). To improve the 
learning process, it is therefore ideal for each learner to have their own personalized scenario. Moreover, 
while learning, some characteristics of the learner may change (e.g. more motivation to learn about a topic 
than another, less time because of personal issues), making the scenario, as defined initially, less and less 
appropriate. It would be difficult for the teacher, particularly in an online context, to detect the change in the 
learners’ characteristics to propose a new adapted scenario. However, this detection is achievable by 
computer-based methods based on the exploitation of learning traces, learner modeling (Greer and McCalla, 
2013) and intelligent tutoring systems (Ma et al., 2014). However, these methods usually require an 
important quantity of traces (hard for courses with few students enrolled) and when new profiles are detected 
the system may need reengineering or a refinement of some parameters. Thus, there can be issues relative to 
the real-time detection of changes in learner profiles to assign them an appropriate scenario. More 
fundamentally, various works on metacognition and self-regulation show that involving the learner, for 
instance by making them choose their learning goals, can lead to deeper learning and increased motivation 
(Harley et al., 2018), compared to a linear more passive way predefined by the teacher. This approach forces 
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the learners to re-evaluate their decision if they realize they have chosen an activity for which they do not 
master yet all the required skills. 

Following these observations, this paper focuses on the co-construction of the learning scenario by the 
learners, as they learn, to make the learning process or acquisition of knowledge more efficient. We use the 
term “co-construction” because although the next learning goal depends on the learner, the range of their 
choice is constrained by the teacher, to prevent them from making illogical choices (e.g. trying to acquire a 
competence before its prerequisite). In this context, our research questions are: (RQ1) Can we set up a model 
allowing each learner to co-construct his or her scenario during the learning process? (RQ2) Is such a model 
understandable and acceptable to learners? (RQ3) How do learners use the possibilities of co-construction 
made available to them? Our contribution is to provide learners with conceptual and technological tools to 
build their learning scenario in a learning context imposed by the teacher and supported by technology. 

The remainder of this paper is organized as follows. In section 2, we present a brief overview of related 
works on personalization and adaptation of learning. Section 3 presents our model of co-construction of the 
learning scenario. Section 4 presents our implementation of the model in the LMS. Section 5 presents results 
of an evaluation of our approach in terms of acceptability of the model by learners, but also an evaluation of 
the system usability through an analysis of data collected in a preliminary experiment conducted in real 
situation with a class of students. We conclude on opportunities for some system redesign ideas. 


2. RELATED WORK 


The description of a learning scenario can be formalized with the Educational Modeling Language (EML) 
(Koper and Manderveld, 2004) which offers the modeling of reusable, interoperable, rich and customizable 
learning units. Through personalization and reuse, it is possible to design several scenarios, but the EML 
language does not provide ways to switch from one scenario to another during the learning. This is because 
the scenario design is generally based on the intentions of teachers (Emin et al., 2010) (teacher-centered 
pedagogy) and on pedagogical goals (Dalziel, 2008) (content-centered pedagogy). Some works have tried to 
be closer to a learner-centered pedagogy, for instance by taking into account teachers' intentions, activities to 
do by learners and learner interactions (Mariais et al., 2010). 

To design a pedagogical scenario, (Esnault and Daele, 2003) defined 17 dimensions of question, taking 
into account learners’ individual differences. However, to take this personalization into account, the scenario 
designer must know the learners’ profiles in advance. Even if new scenarios can be designed by reuse and 
adaptation of existing ones (Riad et al., 2012), profiles can evolve during the learning process and no 
personalized path corresponds to the new profile. (Marne and Labat, 2014) proposed a scenario based on 
activities with several input and output states. The links between activities based on prerequisite relationships 
among them makes it possible to have several learning paths. However, their model, defined in a context of 
serious games, does not give the learner the possibility to choose the scenario to follow. 

The Competence-based Knowledge Space Theory (CbKST) offers a model for structuring 
competences-based learning for personalization (Heller et al., 2006). From the relationship of prerequisite 
among competences, the model constructs several recommended learning paths (Kopeinik et al., 2012). Each 
path is composed of knowledge states (set of competences acquired in a particular field). From a knowledge 
state, the learner progresses in their learning by choosing a competence to acquire that will bring them into a 
new higher knowledge state. The learning is complete when the learner is in the terminal knowledge state 
(state with all acquired competences). Although the CbKST offers several learning paths, it does not consider 
learning constraints (temporal and qualitative related to satisfaction threshold of activities) in choices of 
paths, nor multi-goal activities (e.g. case studies), nor the conditions to change paths (e.g. a change can take 
place after a certain number of failures or the incapability to reach a fixed goal or a temporal constraint 
non-respected). 
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3. CO-CONSTRUCTION MODEL OF SCENARIO DURING LEARNING 


3.1 Core Concepts of the Model 


Knowledge acquisition is rarely linear: there are many ways to do it, depending on the learning goal. But to 
build a scenario, the course design model must allow it. Our model relies on five concepts (cf: Figure 1): 


1. Decomposition of knowledge by learning goal to be achieved. Time constraints are associated with each 
goal. Satisfaction thresholds are added to constrain learners’ efforts. 

2. Encapsulation of knowledge in learning resources for learning goals. This encapsulation guarantees 
modularity in a course since a resource is reusable in another course without modification. 

3. Assessment of acquired knowledge. We define activities to assess the learning. To prevent assessment 
from depending on only one activity, we define for each activity a percentage of participation in 
knowledge validation. An activity can also contribute to the validation of several knowledge. 

4. Prerequisites between knowledge. There can be many ways to learn a course, but there are nevertheless 
order constraints, taken into account in our model by a prerequisites graph between the goals. 

5. Grouping learning goals into learning units. To be close to the teachers’ practice, the goals are grouped 
into learning units (generally parts, chapters, titles, ...). 


Resource 


Evaluated Learning Object e 5 ., Learning Unit 


Learning Object = 
<I +Min Time Learning Goal 

ane +Max Time <J — a 
+Validation Level ; Contribute 


A Activity 
+Assessor 
+Assesssment Type 


Figure 1. Class diagram of learning objects for course design 


+Rate Participation 


Our model constructs several learning scenarios by articulating the learning objects. In a previous work, 
we have also assessed our model acceptability (https://goo.gl/forms/nel Uua4UeYPW3EeO2) from the 
teacher’s point of view (Mbatchou et al., 2018), showing their willingness to use it. An experiment with 16 
teachers from 8 specialties also allowed us to (1) detect and correct the inconsistencies in their educational 
productions; (2) find that certain goals of their course are not related to others; (3) find that there is little 
prerequisite relationship between goals; (4) to note the multiplicity of scenarios in their course. 


3.2 Scenario Building 


The model is meant to provide learners with an environment allowing them to learn the way they want while 
respecting the rules and constraints of learning. We assume that we do not have a priori learner profiles, as 
learners’ profiles are dynamic and we do not want to regularly ask learners to self-report their motivation, 
time, etc. Learners need to build their scenario as they learn. The model is based on knowledge states to 
enable each learner to situate themselves in their learning and to progress. A knowledge state is a state that 
describes acquired and validated knowledge by a learner; it is composed by achieved learning goals. The 
knowledge states are produced and associated according to the Knowledge Spaces Theory to obtain different 
learning paths. The learning process is to guide the learner from initial state to final state. The learning 
constraints defined by the teacher when designing learning objects is an implicit guidance contributing to 
co-construction. Learning is supervised by a human tutor as a learning facilitator (role not detailed in this 
paper). 

During the learning process, the system determines the learner’s knowledge state and offers them a set of 
goals to achieve. Then for the chosen goal, the system proposes a set of resources and activities that will 
allow them to reach it. After an assessment that the knowledge is acquired, the system determines their new 
knowledge state. If they are unable to perform a given activity (resp. progress in a chosen scenario), the 
learner can abandon it and choose another activity (resp. scenario) offered by the system in the same scenario 
(resp. according to the learning goals). 
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The model integrates knowledge assessment modes to progress in learning. The choice of the mode 
depends on the challenge that the learner sets for themselves at any moment. Since the learner is situated in 
learning by their knowledge state, suppose a state with P goals {Gj, G», ..., Gp}. Each G; has a set of learning 
activities {A'|, Ab, ..., A’ni} for validating the acquired knowledge. Each activity A’; has a percentage of 
participation P'; to achieve the goal G;. When a learner chooses to perform the activity A’; we keep the 
obtained value V'; to compute the score obtained for this goal. The validation of each goal (G;) is constrained 
by a threshold (S;). To validate his state with P goals, the learner has the following modes: 

Assessment mode by flexible compensation. The state is validated if YP.4 X41 PiVj > Xf1S;. So, 
learner can progress with few efforts made on certain goals because he can obtain them by compensation. 

Assessment mode by restrictive compensation. With the previous mode, a learner can validate a state 
even with one goal with a very low level of satisfaction. To avoid this case, in compensation mode, the 
learner must make minimum efforts for each goal. The state is validated if []f_, eis Pivi > TTP, Sj. 

Strict assessment mode. This mode allows challengers learners to master all goals of a state before 
progressing. The state is validated if Vi, 1 <i< P, pe Pivi = S;. The quality of the built scenario is better if 
the strict mode is used throughout the learning. 
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Figure 2. Learning process 


4. SYSTEM OVERVIEW 


We chose to implement our model as a plugin in MOODLE (Modular-Object Oriented Dynamic Learning 
Environment), which is the LMS used in our test university (the model being platform-independent). The 
plugin is named EGbKST (Educational Goal based Knowledge Space Theory) and has a dynamic interface 
for learning (cf Figure 3) and a visualization interface of its results that is visible only at the request of the 
learner (not presented here). Learning is organized in dynamic blocks (Communication, Statistics, Resource, 
Goal and Activities) whose content and visibility depends on each learner and their knowledge state. 
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Figure 3. Learning interface 


The learner initially chooses a goal to achieve (block in green). As soon as it is reached, the system offers 
them a new set of goals they can achieve and so on. The learning ends when the learner has achieved all the 
goals. The goals and the order in which they are chosen represent the scenario built by learner. The system 
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allows to change current goal to choose another one if necessary. To progress in learning, the learner has a 
list of assessment modes (block in red) to choose from to express their desired degree of challenge. The 
efforts made and the chosen mode allow it to progress at a higher knowledge state. 

So, we answered positively to our first research question, proposing a model that allows learners to 
co-construct their learning scenario. 


5. ASSESSING CO-CONSTRUCTION OF SCENARIO BY LEARNERS 


5.1 Methodology 


The experiment was realized in 3 phases in a public university in sub-Saharan Africa, with nearly 3500 
students enrolled in 21 academic sections and 120 teachers in 15 specialties (from bachelor to doctorate). 

Phase 1: Assessing the acceptability of the model by the learners. To answer our second research 
question, we submitted a survey to students (https://goo.gl/forms/EgiVdEgE|1z8mfFQr1). The survey 
questions are in affirmative form with responses on a 4-point Likert scale extending from "strongly disagree" 
to "strongly agree". The survey collected student opinion on the following aspects: (1) Current educational 
model: the question is to find if they find that (a) the courses have clearly defined and identifiable educational 
goals, (b) for the defined goals, do they have learning resources and activities to evaluate them? (c) can the 
learning be done in a different order than the teacher’s? (2) Interests for a goal-based educational model: the 
question is to know if they think that a such model would facilitate their learning and success. The 
questionnaire was sent to all 3500 students, but we received only 85 responses (Consulted at 11-24-2017). 
This can be explained by the fact very few students are trained to take online courses (around 250 students 
have access to online training platform). Participants come from 14 academic sections and 3 teaching cycles. 
Their age varying between under 18 years to over 45 (M = 21.60, SD = 6.46). 80% of survey responses are 
from learners who have been trained in the use of online learning platform. 

In view of the response rate, these results should be taken with caution, because it probably over 
represents certain categories of students (e.g. motivated, technophiles). To counter this potential bias, we also 
asked those questions to the 11 students who tested the system (cf, below). 

Phase 2: Assessing the usability of the system. We tested the usability of our system during a real-life 
experiment on the "General Political Economy |" teaching unit taught in the Ist year of the Legal Sciences 
academic section for 2 ECTS credits. Students (NV = 11) are professionals in continuing training whose age 
vary between 24 to over 50 (M = 36.22, SD = 6.38). To carry out this experimentation, the teacher agreed to 
adapt his course according to our model (29 learning goals, 33 learning resources, 31 learning activities and a 
teacher-recommended scenario). Learning takes place over 2 weeks. The resources are a mix of files, 
hyperlinks and videos. The activities are of the production type and quiz (true/false, yes/no, matching, single 
choice and multiple choice). The experience is organized in 2 stages. Stage | took place in a 2-hour 
classroom session during which we explained to learners and tutors how the new teaching model worked. 
Stage 2 consisted to learn online under supervision of tutor. 

Phase 3: Assessing the learning satisfaction of the learners. At the end of the course, learners filled a 
survey (http://foad.uasz. gouv.sn/mod/questionnaire/view.php?id=5274) evaluating their satisfaction with the 
new learning model. The survey questions are in statements with a 4-point Likert scale (from "strongly 
disagree" to "strongly agree"), focusing on the perceived impact of the model on ease of learning and 
contribution to success. 

Assessment of the model acceptability (phase 1) is done with the Google Forms tool, with data saved in a 
CSV (Comma-Separated Values) file. During learning (phase 2), learner interactions with the system are 
recorded in a plain text log file in which each line contains a quintuplet (date, action, object, score, learner), 
corresponding to the action done by a learner on a learning object. The data (CSV file) of the learning 
satisfaction (phase 3) are collected from the Moodle platform of training. 

To validate our third research question, we considered 2 indicators: diversity of scenarios and of 
assessment modes. The diversity of scenarios allows to determine if co-constructed scenarios are different. 
For each learner, we extract successive learning goals followed in chronological order. For those who have 
not completed their learning, we compare their learning sequence with the corresponding sequence in the 
reference teacher-recommended scenario (e.g. the first 5 steps for a learner who dropped out after 5 steps). 
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The diversity of scenarios is represented by the number of different scenarios and the distance between 
alternative scenarios (distance based on the Levenshtein distance - when computing distances between 
scenarios, we only consider sequences of identical length). The diversity of assessment modes allows to 
determine the willingness of each learner to progress according to the mode chosen at each learning stage. 
This indicator is broken down into 2 sub-indicators: the percentage of time that each assessment mode is used 
to progress, and for each mode, the number of learners who used it and the number of times used. 


5.2 Results and Discussion 


5.2.1 Acceptability of the Model by the Learners 


The acceptance of the model is assessed in the general framework with all 85 respondents. We present below 
the results and then contrast them with the results obtained with the 11 students involved in the experiment. 

Current educational model. The survey shows that the courses are organized mainly in chapters 
(81.2%) and often in parts (32.9%). 27% of participants estimate that certain learning goals do not have 
learning resources clearly associated to them. 3.5% of participants believe that in some courses, goals are not 
announced. Results are more concerning for exercises, for which 50.7% learners estimate that educational 
goals are not assessed. This finding justifies our approach to associate resources and exercises with each goal 
to better structure and facilitate learning. 70.6% of participants’ estimate that the course could be better 
learned with a different scenario than the one imposed by the teacher. We conclude that current educational 
model contains weaknesses identified by learners and their wish reinforces our approach of co-construction. 

Interests of pedagogical model based on goals. 81.2% of learners estimate that learning would be easier 
if it is organized and presented by goals and not by chapter. 91.8% of them believe they would obtain better 
results if they were assessed by goal. The results obtained from the 11 students of our experiment are similar 
to those obtained on the larger sample. The only difference is the availability (online) of resources and 
activities for the goals. This difference is justified by the fact that online course procedure requires the 
availability of resources and activities for each learning sequence. 

We thus can respond positively to our RQ2: our approach seems in agreement with learners’ expectations. 


5.2.2 Scenarios Diversity 


To visualize different scenarios followed by the learners, we represented each stage of the scenario of each 
learner with a different color (with gray corresponding to identical steps in teacher-recommended scenario — 
cf. Figure 4). We can see that the learners have built 4 of 11 possible scenarios (called call A, B, C and D), 
where scenario A is built by 72.7% of learners (cf: Figure 5) and corresponds to the one recommended by the 
teacher. We think this high preference rate is related to a system bias, because the proposed goals for the 
choice appeared numbered. Therefore, it seems normal that learners chose the natural order (increasing) of 
goals when they did not have strong preferences. The distance between the scenarios shows that scenario 
C (resp. D) is the most distant (resp. close) to the recommended one (cf. Figure 6). These results show that 
when giving choice to the learners to build their own scenario, they can build a variety of logical scenarios 
while respecting to pedagogical constraints. It should be noted that possible variability of the scenarios was 
limited by the fact that the teacher had chosen to impose the order of chapters. For example, it was not 
possible to move to a goal in chapter 2 as long as all the learning goals of the first chapter were not validated. 
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Figure 4. Visualization of 4 scenarios built by learners Figure 5. Representation Figure 6. Levenshtein distance 
of learners by scenario between different scenarios 
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5.2.3 Assessment Modes Diversity 


Although the strict mode is the most difficult, we found that all learners used it more than 75% of the time 
(cf: Figure 8). This mode is selected by default at the beginning of learning, which can justify that all learners 
are assessed in this mode at least once. Nonetheless, the number of times used shows the desire to remain in 
this mode. This point of view is reinforced by a manual trace analysis which reveals that certain learners 
(e.g. LOS and LO9 in Figure 7) return to certain activities to improve their score to stay in this mode. Except 
for 4 learners (cf. Figure 7), we find that they are challengers (learners who like to validate all activities 
without compensation). We think that once the learners have changed the assessment mode, they want to 
progress quickly and therefore preferred the flexible mode over the restrictive one. We also find that all 
learners who changed their mode stopped some time after and did not view all the educational content. This 
could mean that changing to flexible mode indicates future dropout, and that could be brought to the 
teacher’s attention. 


Educational Unit 02 Educational Unit 03 Educational Unit 04 mleomer = lve 
ot Strict 100! 
2 mode 75,15% 
U3 
Lo4 
Us Restrictive  0,00% 
7 mode 0,00% 
1) 
mT) 
Flexible 36,36% 
rm mode 24,85% 
Figure 7. Scenario representation of each learner by assessment mode Figure 8. Representation of learners and 
(Strict mode in green; Flexible mode in orange) progress number by assessment mode 


5.2.4 The Learning Satisfaction of the Learners 


54.5% (6 of 11) of learners answered to the post-study survey (4 of them reached the third learning unit). 
50% estimate that their learning was facilitated by the initial goal announcement. 66.7% thought the course 
presentation by goal (vs. by chapter) facilitated their learning, confirming the value of our approach which 
propose to structure a course by goal to facilitate its accessibility. 50% said they liked choosing their learning 
path because they are central actors of their learning, confirming our observation that the diversity of 
scenarios is well appreciated and used. However only learner LO7 explicitly declared thinking his success 
was related to our approach. Conversely, learner LO2 said: "This software causes a lot of problem because my 
progress was very slow. The internet connection caused me great prejudice. (...) I propose to let us continue 
with the old method.". The main reason is that our approach requires frequent connection to assess their 
learning to unlock the next learning content, which can be an issue in the sub-Saharan African context. 


5.3 Analysis for Reengineering 


This experiment revealed system weaknesses to correct to avoid bias in the analysis of learners’ behaviors. 

Bias when constructing a scenario. To correct it, we will hide goal numbers and instead present for each 
goal metadata such as duration, validation threshold, description, the number of resources and activities. This 
change should allow for a greater diversity of scenarios and more meaningful choices. 

Decision between challenge and progression. We realized that some learners stay on strict assessment 
mode while they are not progressing. We propose to find a mode that allows them to progress and 
recommend it. This decision is made because we think some learners forgot they could change their mode. 

Choice of assessment mode. To avoid a default mode, we will explicitly ask learner to choose their 
assessment mode at the beginning of the learning. 

Risk of school dropout. Whenever the learner changes their assessment mode to a less challenging 
mode, we will notify the teacher and ask the learner, the reasons of change to better understand their 
motivations. 
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6. CONCLUSION 


Giving learners the opportunity to build their scenario while learning, making them a main actor of its 
co-construction, is not really considered in recent research in TEL. Our model shows that it is possible, and 
that the built scenarios respect educational constraints defined by the teacher. Experiments led with teachers 
and learners show their satisfaction and the ability of the model to improve both the learning and teaching 
processes. The diversity of scenarios built by learners revealed that some learners seem to prefer a different 
approach than the teacher’s default one. Moreover, the model offers learners to modify their assessment 
mode at any time. Their desire to be challenged (strict mode used more than 75% of the time) is a sign that 
our model offers a motivating framework to better acquire competences. This is confirmed by the fact some 
learners returned on previous activities to improve their score to remain in a strict assessment mode. We 
observed that some learners prefer the challenging mode, even if it slowed down their progress. To avoid 
drop-out, we plan to identify this indicator moment and recommend using a less challenging mode. 

Among the limits of this work, the context of our experimentation (few online learners in sub-Saharan 
Africa) does not allow us to fully validate our approach — integration toa MOOC could help reaching a more 
reliable conclusion. Moreover, our model is only applicable for learning by competences or educational 
goals. A first analysis of learning traces allowed us to define reengineering axes, which will give us more 
accurate learning tools and highlight the existing ones. 

In future work, we will integrate into the model the analysis of the chosen scenario and present it to the 
learner. When they face difficulties while diverging from the reference scenario, we may redirect them 
towards the reference scenario. Moreover, traces analysis over several courses could help in identifying 
patterns and thus learner profiles and learning indicators that will help us to guide or redirect future learners. 
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